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Description 



[0001] This invention relates to an electronic image 
processing apparatus which operates continuously and 
which includes means for monitoring its internal image 
processing to ensure correct operation. More particular- 
ly, the present invention pertains to an apparatus where- 
in diagnostic data is added to a stream of data to provide 
a criteria for monitoring the operation of various image 
processor stages. 

[0002] During the manufacture and off-line testing of 
image processing devices, repeatable input data can be 
supplied from a test pattern generator, and subsequent 
analysis of the processed data can allow evaluation of 
the image processing function. However, the operation 
of image processing devices is difficult to monitor on- 
line because they receive real-time, real world inputs 
which are not precisely repeatable. Because the inputs 
are not fixed, analyzing the processed output data from 
such devices based on the input data does not provide 
sufficient assurance that the devices are operating prop- 
erly. 

[0003] An objective of the present invention is to pro- 
vide a means of checking on-line the operation of an 
image processing apparatus. 

[0004] A further objective is to provide an image 
processing apparatus checking means by making use 
of the horizontal blanking times. 
[0005] Yet a further objective is to provide an image 
processing apparatus which performs several sequen- 
tial processing steps on incoming variable data, wherein 
said apparatus includes means for verifying the opera- 
tion of the apparatus at each step. 
[0006] Two dimensional electronic images are tradi- 
tionally transmitted from point to point in a raster fashion, 
with formatting information embedded with the image 
data. The formatting includes a vertical synchronization 
signal indicating the beginning of a two dimensional 
frame, and horizontal synchronization signal indicating 
the beginning of a line. The data for a particular image 
frame is started with a vertical and horizontal synchro- 
nization, and then the data for the first line. Then the 
horizontal synchronization signal is repeated, followed 
by the data for the second line. The horizontal synchro- 
nization signal and line data are subsequently produced 
for each line of the rest of the frame. After the end of the 
last line of the frame, a horizontal and vertical synchro- 
nization are again given to indicate the start of the next 
frame. A time delay is included between the end of the 
data for a given line and beginning of the data for the 
subsequent line. This time delay is known as horizontal 
blanking and is included to allow time for the display and 
other equipment to complete the processing of one im- 
age line and prepare for the next image line. Similarly, 
an additional time delay may be included between the 
end of the last image line of a frame and the beginning 
of the first image line of the next frame. This second time 
delay is known as vertical blanking and is included to 



allow time for the display and other equipment to com- 
plete the processing of one frame and prepare for the 
frame. 

[0007] The horizontal and vertical blanking times in an 
5 image communication have been utilized to transfer 
control and formatting information related to the images. 
Examples of this are program identification and color 
correction standards. Unrelated information has been 
combined with image information to utilize otherwise un- 
10 used communication bandwidth, for example for the 
transmission of videotext. In the present invention, the 
blanking periods are used for the transmission of diag- 
nostic data. 

[0008] The EP-A-0202865 discloses an apparatus 
is provided with a video display generator comprising input 
means for receiving input video composite data, said da- 
ta including an input video image component. The input 
composite data includes sequential input data blocks 
separated by blanking periods to define a two dimen- 
20 sional input video image for display on a video screen 
and an input diagnostic component. The input diagnos- 
tic component is received during said blanking periods. 
The module further comprises image processing means 
for processing input composite data to produce proc- 
25 essed composite data. The processed composite data 
includes a processed video image component formed 
of sequential processed video data blocks separated by 
said blanking periods and a processed diagnostic com- 
ponent during said blanking periods. 
30 [0009] The image processing apparatus of the 
present invention is defined in the claim. 
[0010] Briefly, the image processing apparatus con- 
structed in accordance with this invention includes 
means for acquiring or receiving imaging data repre- 
35 sentative of a variable image and means for generating 
a known pattern diagnostic data which is interleaved 
with variable image data. This combined image data is 
processed within an image processing stage. At the end 
of the processing step or steps, the processed image 
40 data is transmitted to the next processing stage and the 
processed diagnostic data is fed to a separate diagnos- 
tic system and analyzed. The result of this analysis is 
used to verify that the image processing stage is oper- 
ating correctly, or is not correct, give some information 
45 on incorrect operation allowing rapid correction of the 
fault. The known diagnostic pattern of data injected into 
image processor can be varied over time to test the full 
functionality of the image processor stage. The image 
data processing apparatus includes several stages for 
50 processing the variable image in several steps. In this 
type of apparatus, the combined image data is proc- 
essed in multiple steps by the image processing equip- 
ment. At each step in the processing, the processed di- 
agnostic data is captured in a separate system and an- 
55 alyzed, as well as being passed on to the next process- 
ing stage during the blanking time. The analysis of the 
diagnostic data captured between each image process- 
ing stage is used to verify that the image processing ca- 



2 



EP 0 582 546 B1 



3 

pability of each stage is operating correctly, or if not the 
fault can be isolated to a single processing stage. 

Figure 1 shows a block diagram for an image 
processing apparatus constructed in accordance s 
with the present invention; 

Figure 2 shows the combined data stream utilized 
in the apparatus of Figure 1 ; 
Figure 3 shows details of a data acquisition module 
for the apparatus of Figures 1 and 3; and 
Figure 4 shows details of an image processing mod- 
ule. 

[001 1] Referring now to the Figures, an apparatus 1 0 
constructed in accordance with this invention, includes 
several modules, such as a data acquisition module 12, 
and image processing modules 14, 16 and 18. The data 
acquisition module receives (or generates) image data 
which may be for example in the form representable on 
a video screen. 

[0012] For each image to be represented on a video 
screen, the data is partitioned into blocks, each block 
being representative of a horizontal line across the 
screen. Several of these lines are assembled on a 
screen to form a frame. 

[001 3] The sequential arrangement of the data blocks 
is illustrated in Figure 2. Each frame is started with a 
vertical sync signal (not shown). The vertical sync signal 
is followed by the end of a horizontal sync signal 22, 
indicating the beginning of the image data for the first 
horizontal line. At the end of image data period 24 there 
is another horizontal sync signal 26 indicating the be- 
ginning of a horizontal blanking period. In the present 
invention, this horizontal blanking period is used for the 
transmission of diagnostic data 28. The next horizontal 
block is again started with the ending of the horizontal 
sync signal 22 and the processes continues until the 
frame is completed. The end of the frame is signified by 
a vertical sync signal (not shown). The composite data 
stream 34 of Figure 2 is first generated by the data ac- 
quisition module 12 and transmitted over a video chan- 
nel 36 to the first image processing module 14. The 
module 14 performs image processing on the composite 
stream and generates a second data stream for trans- 
mission on channel 38 to module 16. The module 16 
performs its own processing on the stream received 
from module 1 4 and generates another stream for trans- 
mission on channel 40 to module 1 8, and so on. It should 
be understood that in Figure 1 a serial arrangement is 
shown for the various modules, it being understood that 
one or more modules may also be arranged in parallel 
to receive and/or transmit a data stream from a common 
channel. 

[0014] In addition to the image processing step as- 
signed to them, modules also analyze the data stream 
received and/or transmitted and generate correspond- 
ing diagnostic signals 42, 44, 46. These signals are fed 
to a CPU 48 for further analysis if required. 



[0015] As shown in Figure 3, the data acquisition 
module 1 2 could consists of an optical scanner 50 which 
scans an object (not shown) line byline under the control 
of a data processor 52. The scanner 50 then produces 
image data which is fed to a multiplexer (mux) 54. Each 
data line is preceded by a sync signal 22. The image 
data for a line is output from the multiplexer on line 36 
to the image processing module 14. Module 12 also in- 
cludes a register 58 which holds diagnostic data and is 
coupled to the multiplexer 54. This diagnostic data is 
transmitted on channel 36 during each sync signal 26. 
A similar operation occurs at the end of each frame to 
generate a composite data stream 34 as shown in Fig- 
ure 2. 

[0016] Figure 4 shows the elements of the image 
processing module 14. It should be understood that the 
other modules contain and operate in the same manner 
except that they may perform a different image process- 
ing step on the image data. 

[0017] Module 1 4 includes a sync detector circuit 60 
which is used to detect the sync signals of the composite 
data stream 36. When a sync signal is detected by this 
circuit, a corresponding signal is transmitted to a data 
processor 62 which operates the various elements of 
the module. Following the initial sync signal 22, the im- 
age data for a line is provided to an image processing 
stage 64 which performs a preselected image process- 
ing step thereon (such as for example multiplication by 
a constant for amplitude compensation). 
[0018] Following the line image data is the beginning 
of sync signal 26 indicating horizontal blanking. During 
this period the diagnostic data from the composite data 
stream is fed to a register 66. Another register 68 con- 
tains reference signals corresponding to the diagnostic 
signals. The contents of the two registers 66, 68 are fed 
to a comparator 70. The comparator compares the con- 
tents of the two registers and sends the results of this 
comparison to processor 62. The processor analyzes 
this result to determine if the data stream from channel 
36 is acceptable or it has been corrupted. 
[0019] The diagnostic data from channel 36 is also fed 
to image processor stage 64 which performs the same 
operation thereon as the operation performed on the line 
image data. The complete processed data stream is 
sent to a demultiplexer 72 which separates it into a proc- 
essed image data stream and a processed diagnostic 
data stream. The diagnostic data stream is fed to a reg- 
ister 74. Another register 76 contains reference, i.e. data 
corresponding to the diagnostic data from channel 36 
when the image processor stage 64 operates properly. 
The contents of the two registers 74, 76 are fed to a 
comparator 78. Comparator 78 compares the contents 
of the two registers and the results of the comparison 
are fed to processor 62. Processor 62 analyzes this re- 
sult to determine of the image processor stage 64 is op- 
erating properly. The analysis from comparators 70, 78 
are sent to the CPU 48 through line 42. 
[0020] The processed image data from demultiplexer 
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72 are fed to a multiplexer 80. The multiplexer 80 re- 
ceives this image data, the sync signals from processor 
62 and new diagnostic data signals from register 82 to 
assemble a sequential composite image signal on line 
38 for the next processor module 16 where the whole 
process is repeated for its own processing element as 
well as for channel 38. 

[0021] In the above embodiment a scanner was used 
to generate the image data. However, it should be un- 
derstood that for the purposes of this invention, the im- 
age data may be generated by other well known sourc- 
es, or alternatively it may be received from an external 
source. Moreover, the functions of various discrete com- 
ponents (such as registers 68, 76, 82 and comparators 
70, 78) may be performed by the data processors 62. 
Similarly, the functions of the data processor 62 may be 
incorporated into CPU 48. Numerous other modifica- 
tions may be made to the invention without departing 
from its scope as defined in the appended claims. 



Claims 



1. An image processing apparatus comprising : 

(a) an image acquisition module (12) including 
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essed diagnostic component being posi- 
tioned in said blanking periods. 

(ii) reference source means (68, 76, 82) for 
outputting a first, a second and a third ref- 
erence, said first reference corresponding 
to said fixed diagnostic data, and said sec- 
ond reference corresponding to said proc- 
essed diagnostic component; 

(iii) input comparing means (70) for com- 
paring said first reference to said input di- 
agnostic component and for generating a 
corresponding first comparator signal; 

(iv) processed comparing means (78) for 
comparing said processed diagnostic com- 
ponent and said second reference to gen- 
erate a second comparator signal; 

(v) analysing means (62) for analysing said 
first comparator (70) and said second com- 
parator (78) signals to determine if said im- 
age processing means (64) and said chan- 
nel means (36) are functioning properly, 
and 

(vi) output interleaving means (80) for in- 
terleaving said third reference and said 
processed data component to generate an 
output composite signal. 



(i) a source (50) of video image data, said 
image data being separated into image 
blocks (24) separated by blanking periods, 30 
which image data defines a two dimension- 
al video image, 

(ii) an input source (58) of fixed input diag- 
nostic data and 

(iii) input interleaving means (54) for inter- 35 
leaving diagnostic data (28) between said 
video image blocks (24) to provide com- 
posite input data (34) having an input com- 
ponent (24) and an input diagnostic com- 
ponent (28) with said input diagnostic com- *o 
ponent (28) being transmitted during said 
blanking periods; 

(b) channel means (36) for transmitting said 
composite input data (34); 45 

(c) an image processing module (14, 16, 18) 
receiving said composite input data (34) from 
said channel means (36, 38, 40), and including 

(i) image processing means (64) for 50 
processing said composite input data (34) 
to generate composite processed data 
having a processed image component and 
a processed diagnostic component, said 
composite processed data being defined 55 
by processed data blocks separated by 
said blanking periods to form a two dimen- 
sional processed image, with said proc- 



Patentanspruche 

1. Bildverarbeitungsvorrichtung, bestehend aus: 
(a) einem Bilderfassungsmodul (12) mit 



(i) einer Videobilddaten-Quelle (50), wobei 
die Bilddaten in Bildblocke (24) unterteift 
sind, die durch Leerperioden getrennt sind 
und wobei diese Bilddaten ein zweidimen- 
sionales Videobild definieren, 

(ii) einer Eingangsquelle (58) fester Ein- 
gangs-Diagnosedaten und 

(iii) Eingangsverzahnungsmitteln (54) zum 
Verzahnen von Diagnosedaten (28) zwi- 
schen den erwahnten Videobildblocken 
(24), um zusammengesetzte Eingangsda- 
ten (34) zu liefem, die eine Eingangskom- 
ponente (24) und eine Eingangs-Diagno- 
sekomponente (28) haben, wobei diese 
Eingangs-Diagnosekomponente (28) wah- 
rend der erwahnten Leerperioden ubertra- 
gen wird; 

(b) Kanalmitteln (36) zum Ubertragen der er- 
wahnten zusammengesetzten Eingangsdaten; 

(c) einem Bildverarbeitungsmodul (14, 16, 18), 
der die erwahnten zusammengesetzten Ein- 
gangsdaten (34) von den erwahnten Kanalmit- 
teln (36, 38, 40) empfangt, unter Einschluss 
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von 

(i) Bildverarbeitungsmitteln (64) zum Ver- 
arbeiten der erwahnten zusammengesetz- 
ten Eingangsdaten (34), um zusammenge- s 
setzte verarbeitete Daten zu erzeugen, 
welche eine verarbeitete Bildkomponente 
und eine verarbeitete Diagnosekompo- 
nente haben, 

wobei diese zusammengesetzten verar- 10 
beiteten Daten durch verarbeitete Daten- 
blocke definiert sind, die durch die erwahn- 
ten Leerpehoden getrennt sind, um ein 
zwetdimensionales verarbeitetes Bild zu 
bilden, und wobei die erwahnte verarbeite- is 
te Diagnose komponente in den erwahnten 
Leerpehoden positioniert 1st; 
(it) Bezugsquellenmitteln (68, 76, 82) zur 
Ausgabe von ersten, zweiten und dritten 
Bezugsdaten, wobei die ersten Bezugsda- 20 
ten den erwahnten festen Diagnosedaten 
und die zweiten Bezugsdaten der erwahn- 
ten verarbeiteten Diagnose komponente 
entsprechen; 

(iii) Eingangsvergleichsmitteln (70) zum 25 
Vergleichen der ersten Bezugsdaten mit 

der Ein gangs-Diagnose komponente und 
zum Erzeugen eines entsprecheneden er- 
sten Vergleichssignais; 

(iv) Mitteln (78) zum Vergleichen verarbei- 30 
teter Daten, um die verarbeitete Diagnose 
komponente mit den zweiten Bezugsdaten 

zur Erzeugung eines zweiten Vergleichssi- 
gnais zu vergleichen; 

(v) Analysemitteln (62) zum Analysieren 35 
des ersten Vergleichssignais (70) und des 
zweiten Vergleichsignals (78), um zu be- 
stimmen, ob die erwahnten Bildverarbei- 
tungsmittel (64) und die erwahnten Kanal- 
mittel (36) korrekt f unktionieren, und *o 

(vi) Ausgabeverzahnungsmitteln (80) zum 
Verzahnen der dritten Bezugsdaten und 
der erwahnten verarbeiteten Datenkompo- 
nente, um ein zusammengesetztes Aus- 
gangssignal zu erzeugen. 45 



Revendications 

1 . Dispositif de traitement d' images comprenant : 50 

(a) un module d'acquisition d'images (12) muni 

(i) d'une source de donn6es d'images-vi- 
d6o (5), lesdites donnees d'images 6tant ss 
separ6es en des blocs d'images (24) s6pa- 
r6es par des p6riodes sans image, lesquel- 
les donnees d'images definissent une ima- 



ge vid6o a deux dimensions, 

(ii) une source d'entr6e (58) pour I'alimen- 
tation de donnees de diagnostic fixes et 

(iii) des moyens (54) d'imbrication en en- 
tree pour imbriquer des donn6es de dia- 
gnostic (28) entre lesdits blocs d'image-vi- 
deo (24) pour fournir des donnees d'entree 
composite (34) ayant une composante 
d'entree (24) et une composante de dia- 
gnostic d'entree (28), ladite composante 
de diagnostic d'entree (28) etant transmise 
durant les periodes sans image, 

(b) un canal (36) pour transmettre lesdites don- 
n6es composites d'entr6e (34) 

(c) un module de traitement d'image (14, 16, 
1 8) recevant lesdites donn6es d'entr6e compo- 
sites (34) a travers ledit canal (36, 38, 40) et 
ayant 

(i) des moyens de traitement d'images (64) 
pour traiter les donn6es d'entree composi- 
tes (34) afin de generer des donnees trai- 
tees composites ayant une composante 
d'image trait6e et une composante de dia- 
gnostic traitee, lesdites donnees traitees 
composites etant definies par des blocs de 
donnees traces separes par lesdites pe- 
riodes sans image pour former une image 
traitee a deux dimensions, ladite compo- 
sante de diagnostic traitee etant posit ion - 
n6e dans lesdites periodes sans image ; 

(ii) une source de reference (68, 76, 82) 
pour emettre une premiere, une seconde 
et une troisieme reference, ladite premiere 
reference correspondant auxdites don- 
nees de diagnostic fixes, et ladite seconde 
reference correspondant a ladite compo- 
sante de diagnostic traitee ; 

(iii) des moyens de comparaison d'entree 
(70) pour comparer ladite premiere r6f6- 
rence avec ladite composante de diagnos- 
tic d'entree et pour gdnerer un premier si- 
gnal correspondant de comparaison ; 

(iv) des moyens de comparaison traites 
(78) pour comparer ladite composante de 
diagnostic trait6e et ladite seconde r6f6- 
rence pour g6n6rer un second signal de 
comparaison ; 

(v) des moyens d'analyse (62) pour analy- 
ser ledit premier signal de comparaison 
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(78) et ledit deuxieme signal de comparai- 
son (78) pour determiner si lesdits moyens 
de traitement d'images (64) et le canal (36) 
fonctionnent correctement, 

5 

(vi) des moyens d'imbrication de sortie (80) 
pour imbriquer ladite troisieme reference et 
ladite composante de donnees traitees 
pour genSrer un signal composite de sor- 
tie. 10 
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